Integrating Co-Browsing with Other Forms of Information Sharing

ABSTRACT

A co-browse service uses JavaScript to allow a web page shown in a user&#39;s browser to be viewed remotely. Updates to the web page are rendered into HTML and forwarded on the co-browse session. Aspects of the web page that should not be visible are specified in a list of masked elements which prevents the JavaScript from transmitting the content of those elements on the co-browse session. A person viewing the web page at the remote location can select objects to have those objects highlighted within the user&#39;s browser. Likewise the person viewing the web page may manipulate the objects by selecting objects and entering information into the objects. Updates to the web page are collected and aggregated such that only the most recent updates are forwarded on the co-browse session. Updates that don&#39;t affect the DOM, such as hover state, are also transmitted on the session.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International ApplicationPCT/US2013/049615, filed Jul. 8, 2013, which claims priority to U.S.Provisional Patent Application No. 61/776,016, filed Mar. 11, 2013, andU.S. Provisional Patent Application No. 61/673,629, filed Jul. 19, 2012,the content of each of which is hereby incorporated herein by reference.

COPYRIGHT NOTICE

A portion of the disclosure of this patent document contains materialwhich is subject to copyright protection. The copyright owner has noobjection to the xerographic reproduction by anyone of the patentdocument or the patent disclosure in exactly the form it appears in thePatent and Trademark Office patent file or records, but otherwisereserves all copyright rights whatsoever.

FIELD

The field relates to communication sessions and, more particularly, to amethod and apparatus for integrating co-browsing with other forms ofinformation sharing.

DESCRIPTION OF THE RELATED ART

Companies that rely on web sites for sales and other customerinteraction often provide customer support so that, if a visitor ishaving trouble navigating the site, the person is able to interact witha live person such as a customer service agent. The agent may interactwith the visitor using a text based communication session (chat session)or using a voice based communication session (telephone call).

For example, a web site operated by a bank may include a link to a helpservice, to allow potential customers to chat or talk with a customerservice representative to obtain more detailed information aboutfeatures available on their accounts. When the customer clicks on thehelp button, a chat session or voice session is launched, which willcause a customer representative to be connected to the customer.Messages typed into the chat session are relayed between the customerand the customer representative, or the agent and customer have a liveconversation, which enables the customer to describe the problem andobtain assistance from the agent.

When a visitor is having trouble accessing or using a particular aspectof the web site, the agent can try to instruct the customer on how toovercome the difficulty. Unfortunately, the agent's ability to provideassistance may be hampered in this context by an inability of the agentto see the customer's view of the website. Accordingly, from the agent'sstandpoint, it would be advantageous to provide the agent with theability to see what the visitor is seeing.

SUMMARY

The following Summary and the Abstract set forth at the end of thisapplication are provided herein to introduce some concepts discussed inthe Detailed Description below. The Summary and Abstract sections arenot comprehensive and are not intended to delineate the scope ofprotectable subject matter which is set forth by the claims presentedbelow.

A co-browse service allows a vendor's customer service agents to see theweb pages that a customer is currently viewing. Knowing exactly what thecustomer is seeing, the agent is then in a much better position toprovide useful assistance. Privacy is enhanced by masking customersensitive data from the agent's view so that the agent is able toprovide assistance without obtaining access to the customer's privatedata. In one embodiment, ECMAScript such as JavaScript is used to mirrorbrowser state from a visitor to an agent.

A list of masked fields, and potentially divs and pages, is used toprevent the visitor from transmitting information associated with fieldsthat are intended to contain sensitive data to prevent the informationfrom leaving the visitor's browser. Masking a field may be implemented,for example, on an HTML input element by preserving the HTML element onthe agent side but the contents or state of the visitor input to theinput element are not passed to the agent.

The view of the web page depends on (a) the current Document ObjectModel (DOM) represented by a hierarchy of HTML elements as well as (b)state information such as mouse position, scroll offsets, hover andfocus states. To enable the view at the agent to be synchronized withthe view at the visitor, a page's initial DOM tree, as well as anysubsequent changes to it, are forwarded to the agent, but with <script>tags removed. This allows the agent to have a view of what the visitorsees without requiring the agent to have synchronized local stateinformation (e.g. cookies, browser local storage) which would berequired if the agent's browser were to run the same script as thevisitor's browser. In addition, changes that don't affect the DocumentObject Model (DOM), for example in connection with hover state, aredetected and forwarded to the agent separately to enable the agent tosimulate the browser state.

Co-browsing sessions may be established via information available to theagent in multiple forms. Example information systems include chatsoftware, click-to-call customer relationship management software (wherea visitor supplies his phone number so an agent can call him back), andother software systems.

Co-browsing sessions may be combined with other forms of interactiveinformation sharing sessions, including screen sharing sessions, toenable an agent to utilize additional forms of communication whileinteracting with a visitor. A session may also be established during atraditional phone call or third party chat session by verballyexchanging or typing a code that appears when one of the participantsclicks an object on the web page.

BRIEF DESCRIPTION OF THE DRAWINGS

Aspects of the present invention are pointed out with particularity inthe appended claims. The present invention is illustrated by way ofexample in the following drawings in which like references indicatesimilar elements. The following drawings disclose various embodiments ofthe present invention for purposes of illustration only and are notintended to limit the scope of the invention. For purposes of clarity,not every component may be labeled in every figure. In the figures:

FIGS. 1 and 2 are functional block diagrams illustrating the flow ofinformation between participants in a co-browsing customer supportsystem;

FIGS. 3A, 3B, and 4-17 show ways to enable a co-browsing sessionassociated with a visitor to a website to be located by an agent;

FIG. 18 shows an example integration of co-browsing with social media;

FIGS. 19-20 show an example way of highlighting information by an agentto a visitor;

FIG. 21 is a flow chart showing an example process for determiningincremental updates; and

FIGS. 22-28 show an example way of integrating screen sharing sessionswith co-browsing.

DETAILED DESCRIPTION

The following detailed description sets forth numerous specific detailsto provide a thorough understanding of one or more embodiments of theinvention. However, those skilled in the art will appreciate that theinvention may be practiced without these specific details. In otherinstances, well-known methods, procedures, components, protocols,algorithms, and circuits have not been described in detail so as not toobscure the invention.

DEFINITIONS

Customer: A customer of the co-browsing service who subscribes to theco-browsing service. An example customer is the vendor (e.g. the companythat owns vendor website 16) of FIG. 1.

Website: The website owned by the customer that is to be supported viaco-browse.

Group id: A unique id assigned to each customer website that is to beco-browsed.

Visitor: Anyone navigating the website.

Agent: The representative who supports the visitor in carrying outvarious tasks on the website, such as completing a purchase.

Masked field: An input field in a form or on the website that should bemasked from view and not visible to agents. Masked fields, for example,may contain containing sensitive data such as a visitor credit cardnumber.

Masked object: An input field on the visitor's web browser that ismasked from input by agents. Masked objects, for example, may includefields, forms, divs, pages, links, buttons, and other objects that maybe required to remain under the control of the visitor. The agent is notallowed to input, modify, or click on a masked object when running inremote control.

CServer: A secured server that hosts the co-browse sessions, acceptingsession updates from the visitor and relaying them to the agent.

co-browse webserver: A web server controlling access to co-browsesessions by visitors and agents. The co-browse webserver may becollocated with the Cserver or may be a separate entity on the network.

Co-Browse Overview

FIG. 1 shows a network diagram of a co-browsing scenario in which thecontent of a visitor's browser 10 is visible in an agent's browser 12.In the embodiment shown in FIG. 1, the visitor browser obtains a page 14from a vendor website 16. Arrow 1 shows the download of the web pagefrom the website to the visitor. A co-browse service 18 facilitates theco-browse session by relaying web page updates (arrow 2) from thevisitor browser to the agent web browser 12 (arrow 3). To cause thevisitor browser 10 to provide these updates, script 20 (JavaScript inone embodiment) is downloaded to the visitor browser either from thevendor website (arrow 4) or from the co-browse service (arrow 5).Alternatively, the JavaScript may be maintained as a plugin in thevisitor browser. To enable privacy, as discussed in greater detailbelow, a list of masked elements 22 is downloaded to the visitorbrowser. Like the script, the list of masked elements may be downloadedfrom the vendor website (arrow 6) or may be downloaded from theco-browse service (arrow 7). The list of masked elements instructs thescript which elements (e.g. which elements of the DOM) should not betransmitted on the co-browse session. The agent's browser also downloadsa page 24 from the co-browse service (arrow 8) in which the view of thevisitor's browser contents will be displayed during the co-browsesession.

Although an embodiment will be described in which script 20 is describedas being implemented using JavaScript, other forms of scripting languagemay be utilized as well. For example, JavaScript is an implementation ofa scripting language standard by ECMA International in the ECMA-262specification. Other implementations of this standard include JScriptand ActionScript. Thus, although an implementation will be described inwhich JavaScript is used the invention is not limited to this particularimplementation as other forms of script may be used as well.

The co-browse solution enables the agent to view the web page (vendorwebsite) that a visitor is presently visiting. Since the co-browsingsession relies only on JavaScript that is downloaded during the webbrowsing session, the visitor is not required to manually install anysoftware or plugins or dismiss any security warning dialogs in order forthe agent to see the visitor's browser. Likewise, since the JavaScriptis part of the web page served from the vendor website, the JavaScriptwill not carry over to any other web pages, which means that the agentwill not be able to see any pages from other websites, or anything elseon the visitor's desktop.

As described in greater detail below, the co-browse solution enables theagent to see precisely what the visitor is seeing. For example, if thewebpage is not entirely visible to the visitor, the agent will only seethe portion of the web page that is visible. This view will updateautomatically as the visitor scrolls or resizes the page. Where thevisitor has multiple tabs open, the agent is only able to see the tabthat has focus so that the agent views what the visitor is seeing. Theagent can also see the position of the visitor's mouse pointer, whichallows the agent to understand where the visitor is focused whileinteracting with the web page. If the agent's browser is the activewindow on the agent's computer, the agent will also be able to see whichinput field currently has focus on the visitor side. If the visitoropens a select box, that select box will also open on the agent side.

To enable the agent to interact with the visitor in a visual manner, theagent is allowed to click on specific elements or regions of the webpage, on the agent browser 12. When the agent does this, thecorresponding elements will be visually highlighted in both the agent'sand the visitor's browser, so that the agent can direct the visitor'sattention to a specific part of the web page. For example, thehighlighting may take the form of a frame drawn around the specifiedelements, or some other way of visually distinguishing those elements.Preferably the highlighting is visible to the visitor while the agentholds down his mouse button. A few seconds after releasing, or someother specified short period of time it will disappear automatically orfade away without further action by the agent.

Optionally, an agent may be able to provide input to the visitor browser10. For example, the agent may be able to type information into selectedelements on the web page 14 shown on the visitor's browser 10, changethe settings of drop down lists, select radio buttons or checkboxes orotherwise interact with input elements on the page. Depending on theimplementation an agent may be able to click on buttons or links onbehalf of the visitor.

To protect the visitor's sensitive data, such as visitor specific creditcard numbers and other sensitive data, form fields or areas of the webpage containing sensitive data such as credit card numbers are maskedfrom view so that the sensitive data is not provided to the agent. Sincethe data never leaves the visitor's browser, the co-browse service willnever have access to or interact with visitor sensitive data. Maskingdata at the visitor thus avoids requiring the visitor to transmit thesensitive data on the network or otherwise relinquish control over thesensitive data so that the data never leaves the visitor's control.

Optionally, the masked fields may be identified to the visitor to enablethe visitor to know which fields are masked and which fields are notbeing masked. Providing an indication of this nature, such as a greenbox outlining the fields that are being masked, may provide the visitorwith a sense of security by knowing that the information in those fieldsis not visible to the agent and is not leaving the visitor's browser.

Since the co-browse service described below is based on JavaScript,instead of a platform such as Flash, Java, or SilverLight which is notsupported in all web browsers, the co-browse service provides the agentthe ability to view the visitor's browser even if the visitor isaccessing the site using a mobile device, such as a smartphone ortablet. Likewise, the co-browse solution will also work with PCs/Macs.Likewise, uploading the visitor DOM and state information (vs. justsending the visitor page URL) enables the co-browse session to start atany point while the visitor is accessing the vendor website withoutrequiring the visitor to re-create his/her browser's state. Finally,since the only change to the vendor website is the inclusion of thescript, the co-browse service enables agents to provide co-browse basedsupport to visitors with few or no changes to the vendor's website.

Implementation

According to an embodiment, a co-browse service relies on a script tagthat the vendor agrees to insert into the vendor website. ThisJavaScript (e.g. cobrowse.js) may be passed from the web site to thevisitor, or may be passed directly from the co-browse service to thevisitor, when the visitor loads the vendor web page. This JavaScriptcauses the visitor browser, once a co-browse session has been started,to upload the vendor web page HTML and other data from the visitor tothe co-browse service. The co-browse service, in turn, renders this dataas web pages for the agent to view.

Alternatively, rather than having the JavaScript (e.g. cobrowse.js)loaded from the web site to the visitor, a person could manually installthe JavaScript code as a browser plug-in. The advantage of having theplug-in is that it need not be website specific and, accordingly, couldallow the person to co-browse websites which have not been modified toinclude the co-browse script. Specifically, the JavaScript contained inthe plug-in will cause browser state to be forwarded to the co-browsingservice regardless of what web site the person (“leader”) accesses. Thismay be useful, for example, where the leader would like to share theirweb experience with other “followers”, so all can enjoy the socialexperience of visually following along as the leader surfs the web, forexample in connection with social shopping.

In some instances, the vendor's web site might already have otherexisting JavaScript, for example from a chat tool or click-to-callfeature. That existing JavaScript could, with the vendor's permission,be used to invoke the cobrowse.js script. The benefit to this approachis that the vendor could pilot or deploy the co-browse solution with nomodification at all to the vendor's web site.

In one embodiment, the JavaScript (cobrowse.js in this example) does thefollowing either a) at the time a co-browse sessions starts, or b) whena page from the vendor website finishes loading, if a co-browse sessionhas already been started:

-   1. Gets a descriptor of masked elements. A descriptor of masked    elements may be retrieved dynamically from a server at runtime (via    AJAX or websockets), or might be embedded as a JavaScript variable    in the cobrowse.js file, e.g.:    -   Glance.CoBrowse.MaskedElements    -   =[“input#credit_card_number”,”div[masked=‘true’]”]-   2. Periodically:-   a. Clones the document head element, inserts a <base> tag indicating    the base for any relative URL's on the page if a base tag is not    already specified on the page, removes script tags and renders the    cloned hierarchy as HTML. If a <base> tag is already specified on    the page but there are <link> tags with relative URLs that precede    it, those <link> URLs are converted to absolute URLs. Alternatively,    adding the <base> tag and removal of script tags may be performed    server-side by the co-browse service. Offloading these tasks to the    visitor's browser also lets co-browse session servers handle more    sessions.-   b. Clones the document body element, removes script tags and hidden    inputs, replaces the values of masked elements with dummy data, and    renders the clone hierarchy as HTML. Many pages contain one or more    hidden inputs with session state information that does not impact    the page visually. If there is a gesture drawn on the visitor side,    cobrowse.js removes the gesture element as well, since it is    redundant with the gesture drawn on the agent side. Alternatively,    removal of script tags may be performed server-side by the co-browse    service. Offloading these tasks to the visitor's browser also lets    co-browse session servers handle more sessions.-   c. Iterates all user input objects (text fields, radio buttons,    checkboxes, dropdowns) and collects any data input by user. The    current state of canvas objects may also be collected.-   d. Detects visitor mouse position-   e. Detects visitor scroll offsets-   f. Detects dimensions of visitor browser.-   g. Detects which element on the page has focus-   h. Detects whether any select box is open-   i. Sends all of the above (if changed) to the Co-Browse Service.    Data may be sent via an XMLHTTPRequest or XDomainRequest (AJAX), or    Websockets, or some other means of cross-domain communication. Data    sent in this step may be encoded and/or compressed before it is    transmitted. Optionally, if only a portion of the data has changed,    the changed portions only may be sent.    Note that (a) through (h) are repeated periodically, because    JavaScript running in the visitor's browser may make modifications    to the DOM.

Further, in both (a) and (b) script is removed from the HTML that isuploaded to the co-browse service, so that the script does not executewhen downloaded to the agent. If script modifies the document elementhierarchy in the customer's browser, changes to the hierarchy will bereflected to the agent via the co-browse service, so that the scriptdoes not need to run directly in the agent's browser.

Cobrowse.js may be hosted by the vendor or by the Co-Browse Service.This is shown by arrows 4 and 5 in FIG. 1. Likewise, the list of maskedelements obtained by the visitor may be retrieved either from the vendorwebsite (arrow 6) or from the Co-Browse Service (arrow 7). Since thelist of masked elements is specific to the web site, it will be createdby or for the web site to specify a list of elements in the websitestyle sheet that should not be transmitted on the co-browse session. Inone embodiment, the list of masked elements includes a list of CascadingStyle Sheet (CSS) selectors for the website. The list is provided to thevisitor to protect the visitor's private information. In one embodimentthe list is created for each website, although it is also possible tomake a unique list for each page of the web site that the Co-Browserservice on which the co-browsing session is to be implemented.

Selection of elements to be masked may be implemented by allowing anadministrator or other person to visually identify elements of thevendor web site and click on an administrative user interface, e.g. tooltip, to specify whether the visual element should be masked to protectsensitive information or is not required to be masked. Once theadministrator has selected the elements to be masked, a MaskedElementsdescriptor is created based on this input. The Administrative userinterface may be an implemented in cobrowse.js or may be implemented viasoftware provided to the vendor to enable the vendor to select fields ofthe web site to be masked in connection with co-browsing sessions. Toprotect this list, the vendor is required to log in or otherwise beauthenticated prior to making modifications to the vendor'sMaskedElements descriptor.

Although use of a list of masked elements enables a set of elements tobe masked without modifying the website, it is also possible to create amasked field definition which will automatically cause objects with anarbitrary defined attribute to be masked. For example, given a maskedfield definition of *[co-browse hide=true], an attribute such asco-browse_hide=“true” could then be used to identify elements on thewebsite that should be masked.

Differential Updates

Often JavaScript running on a web page modifies the page DOM in responseto various events such as page load, a timer, or a user action. Typicalexamples would be a rotating image marquis where the displayed imagechanges at regular time intervals, or a popup menu which is normallyhidden but made visible in response to a user click.

For efficiency, visitor keeps a snapshot of the entire DOM tree andlooks for differences. The visitor JavaScript, where possible, sendsinformation about the incremental changes to the page to avoidre-uploading the entire page HTML. When an agent first views the page,the agent JavaScript downloads the initial page HTML from the CServerand then applies any incremental updates that arrived since the page wasinitially uploaded. Once the agent is viewing a page, any newincremental changes are pushed immediately to the agent as they arereceived by the Cserver from the visitor.

Frequently, the same elements change repeatedly on the visitor side. Ifthe C Server were to store all of these changes as reported by thevisitor, when an agent first connects to the session the agent wouldreceive a group of updates that all referred to the same portion of thescreen which could cause flashing of that portion of the screen.Likewise, if there is some delay between the time the agent retrieves anew page that the visitor has navigated to, and the time the agent pageconnects to the Cserver to retrieve subsequent updates to the page,multiple changes to the visitor screen may be forwarded to the agent atthe same time causing flashing of the portion of the screen assubsequent updates are applied to the agent view.

Accordingly, in an embodiment the visitor accumulates all differencesand forwards a cumulative difference since the full page has been sent.To generate the set of cumulative differences, the visitor JavaScriptdiscards irrelevant changes which have since been overwritten by otherchanges to the DOM.

Types of Differential Updates

There are three types of incremental differences currently detected bythe visitor: attribute differences (ATTR diff), content differences(CONTENT diff), and HTML differences (HTML diff).

Attribute Difference

An attribute difference indicates that the set of attributes of anelement has changed. Attributes may have been added, modified, orremoved. The ATTR diff specifies the new set of attributes. Optionally,the ATTR diff may specify with finer granularity which attributes havebeen added, modified, or removed.

An example of a JSON representation of an ATTR diff is:

{ ″diff″:″ATTR″,″path″:″#account>div:nth-of-type(1)>table:nth-of-type(1)>tbody:nth-of-type(1)>tr:nth-of-type(1)>td:nth-of-type(1)>p:nth-of-type(1)″,″data″:{″style″:″color:red″, “class”:”boldparagraph”} }In this example, “diff” indicates the type of incremental difference,“path” indicates the css path to the element whose attribute(s) havechanged, and “data” indicates the current set of attributes on theelement. To process an ATTR diff, the agent JavaScript removes all theexisting attributes of the element and applies all of the attributesspecified in the ATTR diff.

Content Difference

A content difference indicates that the text content of an element haschanged. The element must contain no Elements, only a text Node. Anexample of a JSON representation of a CONTENT diff is:

{ “diff”:“CONTENT”,“path”:“#account>div:nth-of-type(1)>table:nth-of-type(1)>tbody:nth-of-type(1)>tr:nth-of-type(1)>td:nth-of-type(1)>p:nth-of-type(1)”,“data”:“somewhere over the rainbow” }In the case of the CONTENT diff, the “data” indicates the new textcontent of the element. To process a CONTENT diff, the agent JavaScriptreplaces the existing text content of an element with the current textcontent specified in the CONTENT diff.

HTML Difference

A third type of difference, an HTML diff, applies if the number, types,or values of child nodes have changed. An example of a JSONrepresentation of an HTML diff is:

{ “diff”:“HTML”, “path”:“#anid”, “data”:“abc <b>def</b> ghi” }Where “data” is the current HTML markup inside the element identified by#anid. Currently when an HTML diff is detected the visitor JavaScriptsends a new full update. Alternatively, some HTML diffs could be handledas differential updates.

Cumulative Diffs

As DOM changes occur, the visitor sends a JSON representation of thechanges to the Cserver. The Cserver forwards the incremental changes toany agents that have already received the visitor's page and have anopen websocket connection to the Cserver. To ensure that the completeset of changes is available to any agents who have not yet retrieved thevisitor's page or have not yet established a websocket connection to theCserver, the Cserver must store the set of cumulative differences thathave occurred since the visitor first sent the page. Often, such as inthe case of the marquis, the page DOM changes many times. Where thisoccurs, the Cserver could end up with an arbitrarily long list ofincremental changes to be applied. An agent joining the session wouldneed to download this long list and apply it. This could perform poorlyand could cause flashing in the agent view. It is common for the samepage element to be changing in the same way each time, for example the“src” attribute of an <img> tag rotates among a series of images,leading to a succession of ATTR diffs for the same element. To preventthe accumulation of incremental DOM changes, the visitor JavaScriptsends not only the most recent incremental change, but also the set ofmerged cumulative changes (a cumulative differential). If the sameelement has the same type of change applied to it, the newest changeoverrides the older change in the cumulative differential.

For example,

{ “diff”:”ATTR”, “path”:”#anelementid”, “data”:{ “style”:”color:red”} }

Followed by:

{ “diff”:”ATTR”, “path”:”#anelementid”, “data”:{ “style”:”color:blue”} }Would be merged into a single cumulative diff:

{ “diff”:”ATTR”, “path”:”#anelementid”, “data”:{ “style”:”color:blue”} }

In an embodiment where cumulative diffs are collected and consolidatedat the visitor, when an agent views the page, the viewer retrieves anddisplays the initial page HTML. The viewer then retrieves the currentset of cumulative differences via a websocket connection, and thenapplies those differences to the view. Any subsequent incrementaldifferences are received via the websocket connection as they arrivefrom the visitor.

An alternative implementation could merge the incremental differences onthe server side. That would eliminate the need for the visitor to uploadthe full set of cumulative differences with every DOM change, but wouldplace greater load on the CServer.

DOM Tree Difference Algorithm

The visitor JavaScript stores a copy of the DOM tree each time it sendsan update to the Cserver. When sending a new update, the visitorJavaScript compares the current DOM tree to the one sent with theprevious update, calculating the set of ATTR, CONTENT, and HTML diffs.The comparison algorithm Diff(elem1, elem2) is shown in FIG. 21. Notethat the Diff( ) algorithm shown in FIG. 21 is a recursive methodinitially called with the document element roots of the previous andcurrent DOM tree as elem1 and elem2 respectively.

In an HTML document, HTML Elements can contain both element and textnode children. Elements are nested HTML elements which may in turncontain other nodes or elements, while text node children contain spansof text. If an Element contains no other Elements (and just one or zerotext nodes), a change in the text in the text node can be represented bya CONTENT diff. If an Element contains both elements and text nodes thenchanges to the text in any of the text nodes would be represented by anHTML change to the parent element.

As shown in FIG. 21, the comparison algorithm Diff(elem1, elem2) (2100)starts by determining if there is an ATTR diff by checking to determineif the attributes of element 1 do not match the attributes of element 2(2102). If there is a difference in attributes, the algorithm creates anATTR diff with the attributes of element 2.

The algorithm next compares the number and types of element children(2104). If the children are different, the algorithm will add an HTMLdiff with the inner HTML of element 2 (2106). At this point, thealgorithm will have detected an HTML difference and accordingly it willstop the recursion (2108) and forward the updated HTML.

If the number and type of element children are determined to be the samein (2104), the algorithm will determine if there are zero elementchildren (2110). If there are no element children, the node is a textonly node. The algorithm will this compare the text content of element 1with the text content of element 2 (2112). If the text of the twoelements is the same, there is no difference between the two elementsand the algorithm will stop the recursion (2114). If there is adifference between the two elements, the algorithm will create aCONTENTdiff with the text content of element 2 (2116).

If there is more than zero element children in (2110), the algorithmwill compare the number, type, and nodeValue of each child node (2118).If these values are different, the algorithm will add an HTML diff withthe inner HTML of element 2 (2106). At this point, the algorithm willhave detected an HTML difference and accordingly it will stop therecursion (2108) and forward the updated HTML.

If the algorithm does not detect an HTML difference at (2118), it willrecurse for each pair of corresponding child elements of element 1 andelement 2 (2120) to look for changes in lower elements of the DOM tree.In this manner, the entire DOM tree will be recursed to look forincremental updates to the DOM which may be used to create updates forforwarding on the co-browse session.

iFrame Elements

A visitor web page may contain iframe elements which reference anotherpage on the customer website. When the visitor Cobrowse.js renders thepage as HTML, the iframe element will be rendered as <iframesrc=“iframurl” /> The contents of the iframe page are not included.However, a separate copy of Cobrowse.js is included in the iframe sourcepage just as with all other co-browseable pages on the customer website.When the iframe page loads, Cobrowse.js in the iframe uploads the iframepage data to the co-browse service as a “sub-session” of the mainsession. Mouse position, gestures, inputs, scroll position, mouse clicksetc. within the iframe are all uploaded to the Cserver as with the mainsession.

When uploading page HTML, the visitor substitutes the iframe srcattribute with a URL to the iframe sub-session. On the agent side, theiframe will retrieve the sub-session view. The agent ifame view willalso contain an additional copy of the agent viewer JavaScriptCobrowseViewer.js, which will update the view continuously based onupdates from the CServer.

Agent View

The agent meanwhile will log into the co-browse service, using astandard web browser, or possibly (for improved performance) a webbrowser enhanced with a plug-in or via some other means. The agent'sbrowser requests the agent view of the customer's page. The page servedto the agent contains an iframe with the same dimensions as the customerbrowser (as shown in FIG. 1). The iframe src URL retrieves a pagecomposed by the co-browse service with the head and body HTML as sent bythe customer, with a <script> tag referencing “cobrowseviewer.js”embedded in it. If the visitor has scrolled the page, the co-browseservice inserts a <div> into the page which is offset from the top leftof the page by the scroll offsets. The div has id “glance scroll” andthe iframe src url includes a relative portion “#glance scroll”, so thatwhen the page loads in the frame it is immediately offset so that the<div> is in the top left of the frame. This approach results in theagent view immediately being positioned at the correct scroll offsetwithout waiting for JavaScript on the page to scroll the agent's view.

Cobrowseviewer.js runs on the agent browser and operates to periodicallyretrieve incremental DOM changes, scroll offsets, mouse position,browser dimensions, element focus, select box state, and input fieldvalues from the co-browse service. The cobrowse service will also notifythe agent if there is a full update available for the page HTML.Cobrowseviewer.js does the following:

-   1. If there is a full update to the page HTML, Cobrowseviewer.js    refreshes the entire frame to get a new view with the current HTML.    (alternatively, the page could rebuild itself using the updated head    and body HTML)-   2. Otherwise, cobrowseviewer.js:-   a. Applies any incremental changes to the DOM-   b. Scrolls the window (which is inside the iframe) to the same    offsets as the customer's browser window;-   c. Displays an image to represent the customer's mouse at the same    coordinates as the customer's mouse cursor; and-   d. Sets the values of the input fields to the specified values-   e. If the agent's browser window is the active window on the    desktop, sets focus to the visitor's focused element-   f. If the visitor has a select box open, opens the same select box    in the agent's view

Gesturing

In one embodiment, actions taken by the agent are captured bycobrowseviewer.js and passed to the co-browse service. The co-browseservice, in turn, relays the captured input to the visitor to highlightaspects of the web site or to take other action in connection on the website 14 displayed on the visitor's browser 10.

FIGS. 19-20 show an example of an embodiment in which, when an agentclicks on an element of the Agent's view of the website, a box is drawnaround the corresponding element in agent's browser and in the visitor'sbrowser 10. In the embodiment shown in FIGS. 19-20, FIG. 19 shows theAgent view. In this example, the Agent clicked on the blank field under“First name” which caused that field to be highlighted in the Agent'sview. Specifically, this caused an orange rounded rectangle 1900 toappear encircling the element in the Agent's view.

FIG. 20 shows the visitor's view of the web site when the Agent clickedon the “First name” field. Specifically, as shown in FIG. 20, when theagent clicked on this object the service instructed the Visitor's viewto draw a rectangle 2000 around the corresponding element in thevisitor's browser. In one embodiment the rectangle drawn in thevisitor's view is colored red. The rectangle change to the visitor DOMis not sent to the Cserver as an incremental update.

Although in this example a red box was drawn when the agent clicked onan element, other ways of visually drawing attention to elements mayalso be used. For example, different colors may be used for the boxes,or a solid semitransparent color may be superimposed over the element tohighlight the element. Optionally, a semitransparent frame may be drawnaround elements in the agent's view as the agent moves his/her cursorabove the agent view so that the agent knows the dimensions of theelement prior to selecting the element to be highlighted.

When an agent selects an element to highlight the element on theviewer's browser, the element may remain highlighted for an indefiniteperiod of time on the visitor's browser. In this embodiment, if theagent clicks on another element, the highlighting will be removed fromthe first element and drawn around the second element. Alternatively,both elements may be highlighted. Likewise, if the agent clicks on ahighlighted element a second time the highlighting may be removed fromthe element. Optionally, rather than having highlighting remain for anindefinite period of time, the highlighting may disappear or fade outafter a period of time. For example, an agent may click on an element tocause the element to be highlighted to the visitor, and then thehighlighting may disappear after a few seconds. The duration with whichan element remains highlighted may depend on the number of times theagent clicks on the element, e.g. one click=3 seconds, two clicks=10seconds, three clicks=indefinite. Many variations are possible dependingon the particular implementation.

Agent gestures rely on the ability to unambiguously identify and accessan element on a web page. In our implementation, when the agent clickson an element to gesture, the JavaScript generates a CSS selector thatuniquely identifies the element. The visitor side JavaScript is thenable to access the element by CSS selector using standard JavaScriptmethods. Some elements on a web page have a unique id, and can beidentified simply by a selector like “#someid”. However many elements donot have an id and must be described by a path through the DOM. Thefollowing algorithm is used to generate a unique CSS selector: Walk upthe element hierarchy, starting with the target element, getting aportion of the CSS selector either as an id or as an nth-of-type(n)selector at each level of the hierarchy. When done, the individual CSSselectors are combined with the CSS child > operator.

The algorithm to produce the ccs selector portion at each level is:

-   -   Get the id of the element (if it has one) and convert it to a        valid CSS id by escaping certain characters or converting them        to unicode escape sequences.    -   If the id uniquely identifies the element within its immediate        parent, the CSS selector portion at this level is #theid.    -   If the id uniquely identifies the element within the entire        document, quit. The element's id serves as a unique starting        point for the path.    -   If the element is the document body, the CSS selector portion is        simply “body”. Quit; the body is the starting point for the        path.    -   Otherwise, determine the index position n of this element within        a list of its sibling elements of the same type. The CSS        selector portion at this level is then tag:nth-of-type(n)

The following provide several examples of unique CSS selectors:

#auniqueidbody>table:nth-of-type(1)>tbody:nth-of-type(1)>tr:nth-of-type(1)>td:nth-of-type(2) #content>div:nth-of-type(1)>form:nth-of-type(1)>table:nth-of-type(1)>tbody:nth-of-type(1)

The agent may also (with or without the visitor's permission) takeremote control. In this mode, the agent's pointer position (which isdistinct from the visitor's pointer) can be displayed to the visitor.The Agent can also type values into fields to input data into fields ofthe visitor's browser and click links, buttons, etc. Cobrowseviewer.jscaptures the key strokes and conveys the input to the visitor's browser.The visitor's browser captures the input in the relevant fields toenable the agent to input data into fields of the web site. This enablesthe agent to provide the customer with direct assistance in navigatingthe web site by enabling the agent to control the visitor's browserduring the co-browsing session.

To prevent the agent from taking particular actions in the visitor'sbrowser, in one embodiment a list of masked objects is defined for theweb site which prevents the agent from entering data into specifiedforms, divs, pages, etc. Likewise the list of masked objects willprevent the agent from clicking on certain links or buttons, such as an“I agree” button on a click-through license agreement. Further, theagent is limited to the particular page and is not able to remotelycontrol pages from other websites.

Reverse Co-Browse

In another scenario the agent may initiate a co-browsing session with avisitor. In this scenario, the agent may open a browser, navigate to aparticular part of the web page, and then launch a co-browse sessionwith a visitor. For example, a visitor is having trouble on a web page.The visitor contacts customer support. Rather than telling the visitorwhere to go, the agent locates the correct site, and launches aco-browse session (using cobrowse.js). The co-browse viewer JavaScript(cobrowseviewer.js) is passed through the chat to the visitor to cause atab or new browser to open on the visitor which connects the visitor tothe web site. The agent can then drive the browser or allow the visitorto control the browser.

Hover State

If a visitor allows their mouse pointer to remain above an element on avendor webpage for a given period of time, the browser may interpretthis action as “hovering”. A web page may be designed such thatdifferent style attributes apply to the element depending on whether itis in a “hover” state. Some forms of hovering result in changes to theDOM, which will be reflected to the agent. For example, JavaScript“onmouseover” may be used to specify the type of effect that should beachieved upon hovering. When an “onmouseover” event occurs, for exampleto display a submenu, this causes a change to the DOM and the normalco-browse DOM update will reflect the change to the agent. However,other forms of hovering do not result in a DOM change. For example, CSSstyles may be defined such that if the visitor's pointer hovers over theelement, a menu of options related to the element will be displayed.This allows additional information to be provided dynamically within apage without requiring the visitor to click on links to move betweenpages.

A problem with this is that the hover state will not affect the DOM, andtherefore the visual changes will not be reflected on the agent sidesimply by keeping the agent DOM synchronized with the visitor. Since theagent's actual mouse pointer cannot be positioned to match the visitor'smouse location, the same hover state does not occur on the agent side,and the agent will not see aspects of the web site that are shown basedon the visitor's hover state. If the visitor side script modifies thedocument element hierarchy in the visitor's browser, changes to thehierarchy will be reflected to the agent via the co-browse service asdescribed above. However, hover state does not change the hierarchy andhence will not be visible to the agent.

Thus, there are instances where the visitor will be provided withfeatures of the web site via hover state events which the agent will notbe able to see. This inability to see what the visitor is seeing maypresent a problem when the agent is trying to help the visitor throughthe web site.

According to an embodiment, an agent that is providing support for avendor web site is provided with a list of hoverable elements, in theform of CSS selectors, for the web site. This list of hoverable elementscan be defined once and used by all agents providing support for thevendor website. The list of hoverable elements may be created in manyways, similar to how the list of masked elements is defined. The list ofhoverable elements is retrieved from the co-browse website along withthe masked field definitions. To enable hover state to be replicated atthe Agent, the mouse position of the visitor is monitored and used toselectively recreate the hover state at the agent side. One way ofimplementing this is described in greater detail below.

The look of a web site is defined using a style sheet. The style sheetdefines styles that apply to both elements and classes of elements. Oneof the aspects that can be defined is to define hover state for anelement/class—i.e. what happens if a visitor hovers their cursor overthe element. For example, a web site may specify that a second levelmenu appears when a user hovers on a first level menu tab, e.g. usingthe following code set forth below in Table 1:

TABLE I .navbar .level1item:hover .level2menu { display:block; }

The agent's host machine actual mouse position does not follow thevisitor mouse. Accordingly, the agent browser does not detect the“hover” state, so the second level menus do not display on the agentside. There are several ways to fix this problem for the agent. Forexample, for the element under the cursor itself the visitor JavaScriptcould call getComputedStyle( ) and send it over to the agent's browser,then have the agent apply that computed style to the same element on theagent side. But that would not work in the case of nested hoveredelements unless getComputedStyle was called not only for the elementunder the cursor but also all elements it contains.

There are two configuration pieces that need to be in place for acustomer whose website relies on :hover styles: 1) a list of “hoverable”elements, and 2) a set of agent CSS styles to be applied to the agentview only. For example, a tool may be created to scan the website stylesheets and generate the list of elements which are hoverable on the website.

A hoverable element is one which has a style that applies to it (or toone of its siblings or children) when the mouse is hovered over it.Hoverable elements may be specified by any valid CSS selector, oftentagname and ID or class. Hoverable elements can be defined on a percustomer basis. Table II shows an example of how website styledefinitions for hoverable elements may be specified:

TABLE II for example, the following style definitions: #div1:hover { /*this style definition applies to any element with id “div1” whenever themouse hovers over the element div1 */ } #div2: hover .c2 { /* this styledefinition applies to an element with CSS class c2 that is a descendentof an element with id div2, whenever the mouse hovers over div2 */ }.c1:hover { /* this style definition applies to any element with CSSclass c1 whenever the mouse hovers over that element */ } h1.c3:hover p{ /* this style definition applies to any <p> elements that aredescendents of an <h1> element with CSS style c3, but only when themouse is hovered over the h1 element */ }The “hoverables” in this example, would be

#div1

#div2

.c1

h1.c3

The corresponding agent styles that are included in the agent stylesheet are shown below in Table III:

.div1_hover { /* style definition matches #div1:hover */ } .div2_hover.c2 { /*style definition matches #div2:hover .c2 */ } .c1_hover {.../*style definition matches .c1:hover */ } h1.c3_hover p { .../* styledefinition matches h1.c3:hover p */ }

To enable hover state to be replicated at the agent, the JavaScript atthe visitor side looks at the mouse position on the visitor side andchecks to see whether the mouse position is located over any “hoverable”elements. If so, the agent needs to apply one (or more) of the hoverstyles to the corresponding element on the agent view. The visitorJavaScript sends a list of styles for the agent to apply. Each entry inthis list consists of:

-   -   csspath: CSS path to the element that is hovered over. The CSS        path is specified using the algorithm described above.    -   classname: the name of the style to apply to the hovered        element. The style name is generated based on the CSS selector        that identified the element. In the example above, if an element        matching selector #div1 is hovered, then the style name would be        “div1_hover”. If an element matching selector.c1 is hovered,        then the style name would be c1_hover. For some websites this        technique might be insufficient and we might need to extend this        naming convention or allow admins to specify the style name that        should be associated with a particular selector.        The agent iterates through this list, finding each element        identified by the csspath, and applying the specified classname        to it, resulting in the same visual effect as the :hover state        style that was applied on the visitor side.

When the visitor hovers over an element, that action is detected and acommand is forwarded to the agent to inform the agent that a hover statehas been triggered. By passing a list of hoverable “CSS selectors” tovisitor, and having the visitor browser determine if anything that ishovered over matches a selector, it is possible to have the JavaScriptpass instructions to the agent to enable the agent to recreate the hoverstate at the agent side.

Co-Browse Service

The Co-browse Service allows one or more customer service agents toview, in real time, the web browsing activity of visitors to a website.Other uses of the co-browse service may be developed as well, and indeedthe co-browse service may be used in any situation where a person wouldlike to view, in real time, the web browsing activity of a visitor to aparticular web site. Accordingly, although an example will be providedin which an agent is able to see what the visitor is seeing, theinvention is not limited to this particular use.

In the embodiment described herein, agents see exactly what visitors seein their browsers, with the exception of the content of designatedmasked fields, such as a credit card number or password. Likewise,dynamic content such as advertisements, which are served from other websites (content from web sites other than the vendor website) will bevisible to the agent, but the agent may not get the same ad as thevisitor.

The co-browse service implements, in one embodiment, several mechanismsthat enable security and enhance integrity of co-browse sessions. Theco-browse website has security measures in place, e.g. login and accountmanagement systems, which enables the website itself to be secureagainst attack.

Architectural Overview

FIG. 2 shows an example architectural overview of an example co-browseservice. The Co-browse service relies on a <script> tag embedded in thecustomer website which references a JavaScript file hosted by theco-browse webserver. For the purposes of this document, this visitorside JavaScript file will be referred to as Cobrowse.js. Other names forthe JavaScript may be used as well.

When a co-browse session starts, Cobrowse.js sends a StartCobrowserequest to the co-browse webserver (Arrow 1 in FIG. 2). Cobrowse.jspasses the customer group id from the visitor 100 to the co-browsewebserver 110, and receives a unique session id, a CServer assignment,and the list of masked fields and hoverable selectors.

The session id, in one embodiment, includes three components: 1) thecustomer group id 2) a session key and 3) a random number foruniqueness. Other parameters may be passed as well depending on theembodiment.

Cobrowse.js stores the session id and the set of masked fields in abrowser session cookie, and proceeds to start uploading session data tothe designated CServer 120 (Arrow 2 in FIG. 2). The CServer in turnposts a message (Arrow 3 in FIG. 2) to the co-browse webserver to recordin the database the fact that the session has started.

Both group id and session key are required to join a session. Oncelogged in, the group id can be determined based on the agent's co-browsegroup membership. For example, the agent will provide support for one ormore customer websites. The JavaScript that is downloaded to the visitorwhen the visitor is at the customer website contains the group id forthe customer, which the customer uploads to the co-browse webserver(arrow 1) when the session is started. This enables the co-browsewebserver to know which customer is associated with a particularco-browse session. Where the agent is supporting multiple customers, theagent can provide the group ID of the customer to the co-browse serverto select co-browse sessions associated with one of the supportedcustomers (arrow 4, FIG. 2).

As for the session key, there are several ways it might be specified onthe agent side. The session key might be a randomly assigned string, ormight be some piece of information associated with the visitor, such asa user id or a tracking cookie id. The agent might enter the session keymanually, or it might be extracted automatically for the agent from datain a CRM record. Additional details of example connection scenarios areset forth below.

Either way, the result is that the agent opens a browser window to a URLassociated with the co-browse session, e.g.:

https://www.cobrowse.net/cobrowse/AgenView.aspx?SessionKey=ssnkey

If not logged in, the agent is prompted to log in and is then redirectedback to the agent view.

AgentView.aspx looks up the session in the database by agent group idand session key, and redirects the agent to the appropriate CServer,which generates the agent view of the session. The agent view includesan embedded JavaScript file, CobrowseViewer.js, which communicates withthe CServer via both HTTP and websockets (arrow 5, FIG. 2).

The CServer sends only HTML markup for the browsing session to the agentbrowser. Resources referenced by the session HTML, such as images orstyle sheets, are downloaded by the agent directly from the customer (orthird party) website.

Security Considerations

The co-browse service collects and transmits potentially sensitivevisitor browsing information, and has therefore been designed withsecurity as the highest priority. The following security considerationsare addressed by the co-browse architecture:

Secure Agent View

When an agent attempts to join a session, the webserver looks up thesession based on the agent's co-browse group and the supplied sessionkey. If a matching session is found, the webserver returns the sessionid, signed using a secret “server key”. The server key is a 256 bitnumber that is known only to the CServer and webserver; a new server keyis generated by the CServer every 60 seconds and shared securely withthe webserver. When the agent is redirected to the CServer to view thesession, the CServer verifies the signature on the session id and onlyallows the agent to join if the signature is valid. This approachguarantees that a request to view a session is only granted if a) theagent authenticated at the webserver and b) is a member of the groupthat the session is associated with.

Once the agent has joined the session, a secure (flagged for HTTPS only)session cookie is used to maintain the agent's session. The agentsession cookie contains the session id signed using a secret key knownonly to the CServer. The CServer secret key is generated at runtime anddoes not persist anywhere.

Websocket connections from the agent also pass the agent session cookievalue when the connection is first established. The CServer only acceptswebsocket connections to a given session if the request includes a validagent session cookie value.

No browsing session data is ever served by the CServer without a validagent session cookie attached to the request. This guarantees that onlyagents who have a registered co-browse provider account with aparticular company are able to view sessions started by visitors to thatcompany's website.

Secure Data Transmission

Using HTTPS or secure websockets for all communication ensures that alldata is transmitted securely, to servers whose identity has beenverified. This includes:

-   -   Downloading the co-browse JavaScript from the co-browse        webserver to the visitor browser    -   Communication between cobrowse.js and the co-browse webserver    -   Communication between cobrowse.js and the CServer    -   Communication between the CServer and the co-browse webserver    -   Downloading session data and the agent JavaScript to the agent's        browser    -   Communication between AgentView.js and the CServer (if the        visitor page is secure)    -   Downloading visitor session resources from the customer website        to the agent browser (if the visitor page is secure)

JavaScript Integrity

Visitor JavaScript is downloaded from the co-browse webserver to thecustomer site (where the customer is serving JavaScript to visitors)using HTTPS from the co-browse webserver. HTTPS ensures that the scriptitself comes from a legitimate source. It is the customer'sresponsibility to ensure that the co-browse JavaScript is embedded withHTTPS into their website, even for non-secure pages. To encourage this,the co-browse webserver will not serve the visitor side JavaScript overHTTP, so that any attempt to access the JavaScript via HTTP should bedetected during testing.

Masked Fields

Masked field definitions cannot be tampered with, as they are stored inthe co-browse webserver's secure database and retrieved over a secureconnection to the co-browse webserver. Alternatively, the masked fielddefinitions may be stored by customers and provided by customers tovisitors as noted above. The contents of masked fields never leavevisitor's browser, so that there is no possibility of this data beingintercepted in transit, or accessed from the CServer even by an agentwith a legitimate session cookie.

Masked Objects

Masked object definitions cannot be tampered with, as they are stored inthe co-browse webserver's secure database and retrieved over a secureconnection to the co-browse webserver. Alternatively, the masked objectdefinitions may be stored by customers and provided by customers tovisitors as noted above. Agents are prevented from providing input tomasked objects to thus prevent the agents from controlling aspects ofthe visitor's browser included in the list of masked objects.Alternatively, a list of controllable objects may be used to specifythose objects within the visitor's browser the agent is able to controlsuch that, by default, an object not included within the list ofcontrollable objects is automatically treated as a masked object and notable to receive input from the agent.

Secured CServer

Each co-browse server 120 is a standalone Unix server running its ownlocal firewall. The firewall is configured to allow incoming trafficonly on secure port 443. No session data persists in any file ordatabase on the CServer. Although FIG. 2 shows the co-browse webserverimplemented separate from the co-browse server 120, otherimplementations may incorporate aspects of the functionality of theco-browse webserver 110 with the co-browse server 120.

No Per-agent Restrictions

Note that there is no specific protection against one agent viewing asession that might have been intended for another agent, as long as bothagents have accounts in the same group. Even when a random session keyis exchanged privately over the phone between visitor and agent, the keyis not protected from snooping because it is stored in an insecurecookie that will be uploaded unencrypted if the visitor navigates to aninsecure page on the customer website. In another embodiment, furtherrestrictions on which agent is able to view a co-browsing session may beimplemented.

Agent Http for Websites Using a Content Delivery Network

If the agent session strictly uses HTTPS to access visitor session data,the agent browser will request all resources such as images and stylesheets using HTTPS. Some website implementations, particularly if usinga content delivery network, cannot deliver all resources over HTTPS.

To work around this, the co-browse service can optionally be configuredto allow the agent protocol to follow the visitor protocol, so that whenthe visitor navigates to an HTTP page, so does the agent, and viceversa. Two separate session cookies are maintained on the agent side, asecure cookie for viewing HTTPS pages, and a non-secure cookie forviewing HTTP pages. While the non-secure cookie provides a minimum levelof security, the agent session is not protected from traffic sniffing.Alternatively, a proxy service may be used to enable the agent to viewwhat the visitor is viewing in this situation.

Co-Browse Connection Scenarios

There are numerous ways for the agent to connect to the correctco-browse session. A given web site may have hundreds or thousands ofvisitors at any one point in time. If the co-browse JavaScript isdownloaded in connection with loading the web page, a co-browse sessionwill exist for each of these visitors. A given agent, who is workingwith a particular visitor, will need to select the correct co-browsesession to enable the agent to see the state of the web site asexperienced by their particular customer.

Accordingly, it is necessary for the agent to identify the visitor andcorrectly identify which co-browse session is associated with theidentified visitor. The particular mechanism that the agent can use todo this will depend on the particular implementation of the web site aswell as what other types of software (e.g. CRM software) are being usedby the Agent.

The agent may be interacting with the customer using a chat serviceprovided by a third party or, optionally, also being provided by theco-browse service. Likewise the agent may have customer relationshipmanagement software running on their workstation that the agent can useto keep track of clients and other contacts. To facilitate interactionbetween the agent and visitor, it often may be advantageous to leveragethese other contacts with the visitor to help identify the co-browsesession on the co-browse service that is associated with the customerwith which the agent is engaged.

FIGS. 3A-3B show an example of how a Co-browse session may be initiatedwhen the visitor needs help (or the agent or automation rules perceivethe visitor may need assistance). In the example shown in FIGS. 3A-3B,the agent and visitor are talking together on a voice call or areengaged in a chat, and the website has a button 300 that will enable thevisitor to initiate a co-browse session. Specifically, if the visitorclicks on button 300 to initiate a session (FIG. 3A) a popup 310 willsupply a random session ID 315 to the visitor (either numeric oralphanumeric). Visitor verbally gives number to agent on call or typesthe numbers into the chat. The agent then provides the number to theco-browse webserver which enables the co-browse webserver to connect theagent to the correct co-browse session.

FIG. 4 shows an alternative to a slider or tab, in which a discreetbutton 400 is placed on a website and used by the visitor to launch aco-browse session. The visitor could click button 400 for help or theagent could direct the visitor to find the button and click it. Clickingon the button would start the co-browse session and show a popup 500 asshown in FIG. 5. Popup 500 is the same as or similar to popup 310 shownin FIG. 3. The visitor can then text the code to the agent or tell theagent the code. The code can be generated by the co-browse service orgenerated in the Visitor's browser. In either case it is associated withthe session and needs to be communicated from the visitor to the agent,unless there is another identifier available on both visitor and agentside.

FIG. 6 shows the agent screen when an agent joins a co-browsing sessionthrough an interface provided by the co-browsing service. As shown inFIG. 6, when the agent would like to join a co-browsing session, theagent will click on an icon on their computer which will show the popupshown in FIG. 6. The agent then types the number provided by the visitor(see FIGS. 3B & 5) to join the session. As shown in FIG. 6, the popupincludes field 600 in which the agent can type the session key to enablethe agent to join the co-browse session with the visitor.

FIG. 7 shows an alternative to typing the number into a popup.Specifically, in FIG. 7 the co-browse service interface provides a listof co-browse sessions to the agent (to which this agent has access, byvirtue of permissions, skill-set or some other criteria) which the agentmay join. The agent selects the co-browse session from the list or typesthe session key or other information into a search field 700 to morequickly locate the correct co-browse session from a set of availableco-browse sessions.

Instead of providing an identifier to the visitor and then requiring thevisitor to provide the identifier to the agent, in another embodimentthe visitor is prompted to input identifying information that can beused by the agent to find the visitor's co-browse session. For example,as shown in FIG. 8, when the visitor clicks on a tab to initiate aco-browse session the tab may provide a prompt 800 instructing thevisitor to enter the identifying information and a field 810 in whichthe visitor is able to input the identifying information. The agent asksthe visitor their name and uses the name to select a session by typingthe visitor's name into box similar to box 600 (see FIG. 6) or to searchfor a co-browse session from a list of current sessions (see FIG. 7).Although the example shown in FIG. 8 requested the user to enter theirname, other customer information may be used instead, such as thecustomer's location, account ID, email address, or other informationavailable in the context of the visitor web session.

An agent may be running customer relationship management (CRM) softwaresuch as CRM software available from SalesForce™. The agent may use theCRM software to locate co-browse sessions as well. FIG. 9 shows ascreenshot of a portion of a CRM software interface. As shown in FIG. 9,the agent can find a lead or contact manually within the CRM systemusing information provided by the visitor. Alternatively, when thevisitor enters information into field 810, if a record exists for thevisitor in the CRM system, the lead/contact screen/page could alert theagent that the lead/contact is in-site and/or requesting help.

FIG. 9 shows an example CRM software interface when the agent types in avisitor name. Specifically, in this figure, the agent will request thevisitor's name and then type the visitor's name in search field 900. Asshown in FIG. 10, the CRM software will populate a list 1000 of contactsand/or leads which match the search. The agent can select one of theentries from this list (e.g. Vera Visitor) to pull the contact recordfor the visitor within the CRM system.

FIG. 11 shows an example contact record for the selected visitor. Asshown in FIG. 11, the contact record includes a button 1100 that theagent can click to join the co-browse session with the visitor. A screensharing button 1110 is also provided to enable the agent to engage thevisitor using a screen sharing session to enable the agent to showinformation back to the visitor. Joining the co-browsing session throughthe CRM record enables the CRM system to keep track of co-browsingsessions within the context of other contact data, to enable analyticsto be built from the co-browsing session information.

Optionally, the CRM system may automatically create a new contact recordor other CRM record for new visitors. An implementation of integrationbetween a CRM system and screen sharing system is described in greaterdetail in U.S. patent Ser. No. 13/624,951, filed Sep. 23, 2012, entitled“Integrating Screen Sharing Sessions with Customer RelationshipManagement”, the content of which is hereby incorporated by reference.In this referenced patent, a mechanism is used to enable the CRM systemto detect initiation of screen sharing sessions and poll a screensharing service for information about the screen sharing session, suchas the duration of the screen sharing session and information about theparticipants to the screen sharing session. A similar mechanism may beused to enable the CRM system to capture co-browsing session.Specifically, the CRM system may determine that an agent has elected toparticipate in a co-browsing session. At a later time the CRM system maypoll the co-browsing server to obtain details about the co-browsingsession, such as the duration of the co-browsing session, theparticipants to the co-browsing session, the pages visited during theco-browsing session, and any other information available to theco-browsing service. Likewise, the co-browsing session may be recordedfor training purposes as well as for auditing.

FIG. 12 shows another example, in which the CRM software has a separatetab showing a list of visitors on the site. In this example, the CRMsystem pulls information and additional fields from CRM records, andpresents available co-browse sessions in list format. The list may beoptimized, for example, by highlighting visitors that are requestinghelp and/or sorting these visitors to the top. For example, as shown inFIG. 12, a list 1200 of visitors in the site is provided. One of thevisitors 1202 is listed as anonymous because the CRM system does nothave any information about that visitor and/or the visitor has notprovided identifying information to the site.

Some web sites provide chat (text based) customer support. It iscustomary for the visitor to be asked to provide identifying informationin connection with starting a chat session. The initial informationgathering which takes place in connection with starting a chat will bereferred to herein as a pre-chat survey.

The particular manner in which a co-browse session may be identified andlinked to a chat session depends on the level of integration between theco-browse service and chat service. If there is integration between thechat and co-browse services, the information from the pre-chat surveymay be used to perform a lookup for a matching co-browse session. FIGS.13A and 13B show example pre-chat survey windows that may be used tocollect information from a visitor when the visitor elects to engagecustomer service via a chat session.

As shown in FIGS. 14A and 14B, the agent will see the visitorinformation in the chat application and may use the visitor informationto look for a co-browse session using the mechanisms described above inconnection with FIGS. 9-12. If the visitor's name or other identifyinginformation is not collected using a pre-chat survey, this informationmay be collected by the agent during the chat session and used to locatea co-browse session associated with the visitor.

FIGS. 15A and 15B show example CRM interfaces that may be used if thechat and CRM systems are integrated with the co-browse service. As shownin FIG. 15A, the contact record for the visitor may be automaticallydisplayed when the agent is in a chat session with the visitor. The CRMrecord, in this embodiment, includes a co-browse button 1500 that theagent can use to connect to a co-browse session associated with thevisitor. FIG. 15 also shows the agent status information about thevisitor, such as the amount of time the visitor has been on the site1510 and the page where the visitor is located on the site 1520. Thisallows the agent to have an idea of where in a transaction the visitoris located, so that the agent has a better understanding of what issuesthe visitor is likely to be encountering.

FIG. 15B shows the agent screen when the visitor is not being tracked inthe website by the CRM system. As shown in FIG. 15B, if the agent wouldlike to join a co-browsing session with the visitor, the agent may beprompted to enter identifying information such as the co-browsing key(see FIG. 3B), visitor's name, etc.

FIG. 16 shows a scenario where there is no integration between the CRMsystem and the chat system, but the CRM system does have knowledge ofavailable co-browse sessions. As shown in FIG. 16, the agent can use theinformation provided in the pre-chat survey or other informationobtained during the chat session to look up a contact record for thevisitor in the CRM system. In the example shown in FIG. 16, thevisitor's location (Denver) was used to search the list of existingco-browse sessions. The agent alternatively could have used thevisitor's name or other identifying information to search for apotential matching co-browse session.

FIG. 17 shows an example agent screen for a chat service. As shown inFIG. 17, the agent may be provided with the ability to join a co-browsesession through a button 1700 in the chat agent's screen. The chatservice provides this agent view to enable the agent to interact withthe visitor on the chat session, without asking the visitor to doanything. In the embodiment shown in FIG. 17, the agent can click onbutton 1700 to automatically join the co-browse session associated withthe visitor that is taking part on the chat session. The agent mayinitiate the co-browse session with no visitor action. Optionally, theagent may offer the visitor the opportunity to participate in theco-browse session, and provide the visitor with the ability to acceptthe co-browse session.

In one embodiment, to enable a chat service to track visitors to a website, the chat service will include script on the web site. This chatservice script will cause the visitor's browser to periodically poll thechat service to determine if there has been a change of state. Forexample, a rule at the chat service may cause the chat service to servea popup or overlay to the website asking the visitor if they would liketo initiate a chat session.

The chat service has the ability to insert additional script into thevisitor's browser. According to an embodiment, the co-browse script isdownloaded to the visitor by the chat service to enable a co-browsingsession to be initiated. The co-browse script may be downloaded based ona rule at the chat service, such as whenever a user selects an item forinclusion in a shopping cart, or when a chat agent selects a co-browsebutton to initiate a co-browse session with the visitor. Many ways oftriggering the download of co-browse script may be used to cause theco-browse script to be provided to the visitor. Optionally informationto be used by the co-browse server to identify the co-browse session(e.g. visitor ID or session ID) may be included in the co-browse scriptthat is downloaded to the visitor. Thus, a chat session need not bestarted for the chat service to download co-browse script to thevisitor.

In another embodiment, when the visitor starts a chat session JavaScript(e.g. cobrowse.js) is downloaded and installed into the visitorsbrowser. In this embodiment, the JavaScript contains an instruction forthe visitor's browser to poll a session status server with anidentifying value. The session status server may be the chat serviceserver, the co-browse server, or another server on the network. Theidentifying value, in one embodiment, is uniquely associated with thechat session between the agent and the visitor.

If the agent clicks on co-browse button 1700, the agent co-browser willinitiate a co-browsing session to create state associated with theidentifying value. This will cause the server session status at thepolling server to be changed to indicate that a co-browsing session hasstarted and will identify the correct Cserver which will host theco-browsing session. When the visitor's browser detects that theco-browsing session has started, it will send browser state to theCserver which will be forwarded to the agent. This will enable the agentto see the view of the visitor's browser. The agent will wait until thevisitor connects to the session and then the agent will connect to thesession to start receiving data.

Since the JavaScript is downloaded to the visitor in connection withscript provided by the chat service, the co-browse JavaScript will causethe visitor's browser to be visible to the agent without requiring thevisitor to take any further action—the visitor is not required to clickon a button to indicate that they would like the agent to be able to seethe browser. To protect the security of the visitor's information, theJavaScript that is downloaded on the chat session also includes a listof masked items to prevent sensitive information associated with themasked fields from being forwarded to the agent on the co-browsesession.

Although the preceding description was provided in connection with achat based communication session, similar location and joiningco-browsing sessions may occur in connection with voice-basedcommunication sessions. Specifically, if the visitor clicks on a buttonto initiate a telephone call, e.g. a “click-to-call” button, and entersa phone number, the identifying information provided by the visitor maybe used by the agent to locate and optionally initiate a co-browsingsession with the visitor.

Polling Minimization

There are instances where parties may wish to communicate via acommunication session, such as a co-browse session, screen sharingsession, Voice over IP session, or web conferencing session. Where oneparty starts the session, and the other party does not know of thesession, the other party needs to be provided with information to enablethe other party to join the session. As noted above, multiple types ofcommunication sessions may experience this problem. Although an examplesolution will be provided in connection with co-browsing, similarsolutions may be utilized with the other forms of communicationsessions.

For example, as noted above, it is possible for an agent to initiate aco-browsing session with a visitor, and then have the visitor join theco-browsing session to enable the agent to view the visitor's browser.In this situation, the agent may initiate a co-browsing session toenable the agent to view what is being shown on the visitor's browser.However, the visitor does not know that the agent has initiated aco-browsing session and, hence the JavaScript at the visitor does notknow to start forwarding information about the content of the visitorbrowser.

In this situation, it may be possible for the JavaScript at the visitorto periodically poll the connection server to determine if a co-browsesession has been initiated by an agent. Continuous polling of theco-browse webserver enables the co-browse webserver to periodicallynotify the visitor as to whether a co-browsing session has beeninitiated by the agent to enable the JavaScript to learn the identity ofthe server hosting the session and enable the JavaScript at the visitorto begin forwarding updates on the co-browse session.

However, for the co-browse webserver, using continuous polling (or rapidpolling) is resource intensive. For example, if the co-browse webserveris affiliated with several websites, each of which has thousands ofvisitors, even if each visitor only polls the connection webserver onceper second, the co-browse webserver would still need to handle manythousands of transactions per second just to handle status updates forvisitors who are likely to have no pending co-browse session.

As the number of identifiable visitors to the website increases, havingthe visitor JavaScript continuously poll for the availability of aco-browse session does not scale. While the rapidity at which thevisitor polls the co-browse webserver may be decreased, this introducesa potential time lag between when the agent selects a co-browse sessionand when the agent first starts receiving data from the visitor.

There are several ways to fix this. For example, as noted above, it ispossible for the visitor to click on a specific button or other featureof the web site associated with the co-browse server (e.g. a buttonincluded on the web site). This enables the visitor to poll theco-browse webserver but is limited to an implementation where thewebsite has a feature associated with the co-browse service.

According to an embodiment, the co-browse JavaScript includes a pollmodule designed to capture specific actions by the user and interpretthese actions as a trigger to poll the co-browse webserver to determinewhether a co-browse session has been initiated. Causing the visitor topoll the co-browse webserver when the user performs a specific actionreduces the frequency with which the visitor will poll the co-browsewebserver, which hence reduces the number of poll transactions theco-browse webserver must handle.

For example, assume a visitor is at a web site and is being helped bythe agent. The agent, in this scenario, clicks on a button or otherfunction to initiate a co-browse session. To enable the visitor to pollfor the co-browse session, and hence to enable the visitor to find theco-browse session, the agent can instruct the visitor to take an unusualaction on the web site such as left or right clicking on a blank area ofthe web site, shake their mobile device, cause their mouse to do threecircles, select a particular letter or other character, or to take anyother unusual action. The JavaScript at the visitor is configured tolook for the specific action specified by the agent. When the JavaScriptdetects the specific action, it causes the poll trigger module totransmit a poll to the co-browse webserver. The co-browse webserverreceives some piece of identification information from the poll, such asa visitor token, visitor ID, or other value, and searches for aco-browse session associated with that value. As noted above, the agentin this scenario knows the visitor and, hence, is able to specify thevisitor token, visitor ID, or other value when the agent started theco-browse session. The co-browse webserver uses the visitor token,visitor ID, or other value to find the co-browse session started by theagent and provides identifying information to the visitor (such as theidentity of the Cserver handling the session). The visitor then startstransmitting data on the co-browse session so that the agent can viewthe web page being shown in the visitor's browser.

By way of example, assume a visitor is logged into a web site such as aretailer's shopping cart, a bank website, software as a service, gaming,or using a common universal login such as Facebook or Google. Thevisitor and agent are communicating by chat or phone, and the agenthence knows the identity of the visitor. The agent may find the visitorin the agent's database (e.g. CRM system) and obtain a visitor's uniquetoken data. Alternatively the token may be created when the visitorcontacts the agent to initiate communication with the session. In anyevent, both the agent and the visitor have the same token. The agentinitiates a co-browse session specifying the token to the co-browseservice. The visitor performs an “unusual” action to cause theJavaScript to poll the co-browse service for the existence of aco-browse session associated with the token. The co-browse servicematches the poll with the pending session and enables the agent andvisitor to be connected to the communication session (co-browse sessionin this example).

There are many types of unusual events that may be detected. Forexample, on a mobile device or other device with a motion sensor,JavaScript in the browser can detect shake events usingDeviceMotionEvents, which is part of the HTML5 working draft. There isno specific “shake” event, rather a shake would be defined as a specificpattern of motion reported by the DeviceMotionEvents. Likewise, not allbrowsers necessarily support the events yet and not all handle thecoordinates the same way, but depending on the implementation a shakeevent could cause the JavaScript to poll the co-browse webserver. Inthis implementation, an agent who is talking or chatting with a visitorwould instruct the visitor to shake their phone/other device. When thevisitor shakes their phone, the JavaScript running on the visitor'sdevice will poll the co-browse webserver to enable the visitor to becomevisible to the agent.

In another embodiment, the visitor and agent need not even becommunicating with each other prior to the shake event. In thisembodiment, the co-browse JavaScript is running in the visitor's browserbut the co-browse JavaScript has not initiated a connection (poll) tothe co-browse webserver. A co-browse session is initiated for eachidentifiable visitor, but an agent is not associated with the co-browsesession. The visitor has been instructed, however, that help isavailable on the website and that the visitor can shake their phone tosolicit help. When the visitor shakes their phone/device the JavaScriptdetects the shake event and initiates a connection with the co-browsewebserver. This causes the visitor to join the co-browse session and anagent supporting the website receives a notification that a visitorwould like help. The agent is connected to the co-browse session and cansee what the visitor is viewing. The visitor is identified to the website and this identification is used to enable the visitor and agent toalso interact via voice or chat using one of the other methods describedherein.

Other unexpected events, other than shake events, may likewise becaptured. Example unexpected events, which may be captured by theco-browse JavaScript and used as a trigger to send a connection requestto the co-browse webserver include single or double clicking (or singleor double tapping on a touch sensitive screen) on a region of the webpage not affiliated with a link. For example, the visitor may beinstructed to click/double click on a blank portion of the web site.When this occurs the JavaScript running in the browser would capture theevent and interpret the event to cause a poll to be sent to theco-browse webserver.

Another unexpected event may be a pinch motion followed by an unpinchmotion on a touch sensitive screen. The user may be instructed to pinch(to cause the web page to zoom out) and then to unpinch (to cause theweb page to zoom in). This dual action, within a specified time periodsuch as within a second, may be used as a trigger.

Another unexpected event may be to rotate the mouse in circles. On atouch sensitive screen, a similar gesture on the screen may be capturedwhen the user touches the screen and draws a series of circles on thescreen, draws a particular letter on the screen, does a diagonal swipe,or otherwise performs a particular gesture which has infrequent useduring normal navigation of a website.

It may be expected that visitors may happen to perform the unusualaction occasionally when not intending to initiate a co-browse session.However, if the action used as the trigger is sufficiently unusual, thenumber of connection requests forwarded to the co-browse webserver willbe significantly reduced compared to having all visitors to a websiteperiodically poll for the availability of a co-browsing session.

Social Media

Social Media sites such as Facebook enable users to share content suchas pictures, text, and other updates of interest. A person has a feed,which is a running update of items that they add to their personal page.According to an embodiment, a co-browse session access link may beposted on the person's feed, which will allow other people with accessto their feed to join the visitor and participate in a co-browse sessionwith the visitor. For convenience an implementation will be described inwhich the social media site is Facebook™, although the conceptsdescribed herein may be used with other social media sites as well. Thisenables many social applications, such as social shopping in which usersco-browse with each other while shopping at websites available on thenetwork.

In one embodiment, a user has a Facebook app installed in their Facebookprofile which will enable co-browse sessions to be added to theirFacebook feed. When the user encounters a website that is co-browseenabled (the website contains the co-browse JavaScript), or if the userhas a plugin containing the co-browse JavaScript, the user may beprovided with an option to post a co-browse link on their Facebook feed.The co-browse link contains a pointer to the co-browse server hostingthe co-browse session as well as a session identifier that may be usedto identify the co-browse session at the co-browse server. When a friendclicks on the co-browse link, a browser window is launched whichoperates in the manner described above in connection with the agent, toenable the Facebook friend to join the co-browse session. Additionalfriends (followers) can also join and watch (the leader). If the leaderenables remote control, followers can take turns controlling where theleader's browser goes.

FIG. 18 shows an example post to a feed in a social media website. Asshown in FIG. 18, the post 1800 includes a link 1810 to a co-browsesession that may be used by anyone with access to the post to join aco-browse session associated with the owner of the feed. For example, ifa person is shopping and would like fashion advice, the person couldpost a co-browse invitation to their Facebook feed to invite theirfriends to co-browse with them while they shop for clothes. Although theco-browsing link 1810 in FIG. 18 is shown as being posted to a feed,alternatively the co-browsing link 1810 could be sent via text messageor email to an individual or targeted group to allow those individualsreceiving the link to join the co-browsing session or to join ascheduled co-browsing session at some point in the future.

In another embodiment, visitors to a web site are identified on the website and compared with a list of Facebook friends, Outlook contacts, oranother list of known acquaintances. The visitor is then presented withan indication that one of their Facebook friends or Outlook contacts isconcurrently visiting the web site, and provided with an opportunity tolaunch a co-browse session with the acquaintance.

As another example, a visitor to a site may be invited to join otherusers also on the site to co-browse with those users to enable communalbrowsing.

In yet another example, a co-browsing session may be initiated whenevera person puts an item in their shopping cart. The co-browsing sessionmay persist after the visitor leaves the site and an email or textmessage may be sent to the user with a co-browse invitation that theuser can use to return to the site or to the last page where the userwas located within the site.

In another embodiment, a visitor on a site with Facebook integration maybe provided with a sidebar list of friends (friends who are theirFacebook friends) who are also on the site. If a user clicks on one oftheir friends a co-browsing session with the friend (or an option tojoin a co-browsing session with the friend) may be provided to thevisitor (e.g., by initiating a Facebook chat with aclick-to-view-my-browsing link).

Co-Browse and Screen Sharing Integration

There are instances where it would be advantageous to integrate thebenefits available from screen sharing with co-browsing. For example,when an agent is viewing a co-browsing session with a visitor, it may bedesirable to provide a live video feed of the agent to the visitor.Likewise, the agent may want to provide the visitor with a video feedavailable to the agent from another device connected to the Agent'scomputer, such as the live video from the screen of a connected tabletcomputer, smartphone, document camera, etc. According to an embodiment,the connection achieved between the visitor and agent to initiate theco-browse session is leveraged to connect the two participants into ascreen sharing session over which content from the agent is provided tothe visitor.

Providing the ability to incorporate screen sharing sessions into aco-browsing session enables an escalating interactive experience tooccur when a visitor initiates a chat session or makes a phone call toan agent associated with a web site. What starts out as a chatsession/phone call to customer support may escalate to a co-browsingsession in which the agent is able to see the visitor's view of the website. The agent may then use screen sharing technology to provide livecontent to the visitor which is selected for the visitor based on thevisitor's location on the web site. For example, to enable the visitorto have a human connection with the agent a live view of the agent maybe provided to the visitor, e.g. in a tiny inset within the web site orin area 310 of the floating tab 300 in FIGS. 3A and 3B. The live feed ofthe agent may be captured by a camera and provided via screen sharingsoftware to the visitor to be displayed to the visitor. As theinteraction between the agent and visitor continues, the agent mayselect other content to be provided to the visitor. This additionalcontent may be a demonstration that the agent launches on their screenwhich is shared with the visitor via a screen sharing session or may bea selection of another live video feed.

As noted above, the Agent has the ability to see the visitor's webbrowser by virtue of the co-browse session. The agent does not, however,have the native ability to see other things which appear on thevisitor's screen since the visitor is not implementing screen sharingsoftware. To enable the agent to see the content that is being providedto the visitor, in one embodiment the agent is automatically joined as aparticipant to the one or more screen sharing sessions. This allows theagent to not only be the source of the content, but also a consumer ofthe content, that is being shown on the screen sharing session. This maybe useful, for example, where a live video of the agent is beingprovided to the visitor on a screen sharing session. By causing theagent to be included as a guest in the screen sharing session the agentmay be provided with a representation of the video that the visitor isbeing provided so that the agent knows exactly what the visitor isseeing.

Integrating screen sharing with co-browsing may be useful in manycontexts, including sales, customer support, technical support, etc.Screen sharing may be implemented using the AJAX viewer described inU.S. patent Ser. No. 13/239,757, filed Sep. 22, 2011, entitled Methodand Apparatus for Facilitating Participation in a Remote ViewingSession, the content of which is hereby incorporated herein byreference.

One advantage of using this combination, is that a two-way webconference is enabled without requiring the visitor to download andinstall any software. Specifically, the co-browse does not require anydownload or Java/Flash/Silverlight or plug-in on the visitor's side.Just as valuable, the AJAX viewer screen share technology lets the agentshow things to the visitor, again without requiring any download to thevisitor. Thus, unlike traditional two-way web conferencing, where bothparticipants need to have installed software, this combination lets theAgent see and show without any download to the visitor. Moreover, itworks for any modern browser, including mobile devices, since it doesnot rely on Java or Flash which may not be supported in particularbrowsers.

Although an example was provided in which a co-browse session wasescalated to a screen sharing session, the opposite order is possible aswell. In this embodiment, an agent launches a screen sharing sessionwith a visitor. For example, the visitor and agent may be communicatingvia a chat session or by voice, and the agent may launch a screensharing session to enable the visitor to obtain content provided by theagent. During the course of the screen sharing session, the visitor maybe asked to enable the agent to see what is being shown in the visitor'sbrowser. The visitor may then initiate a co-browse session to show backto the agent what is currently being displayed in the visitor's browserusing any of the connection methods described above. This allows anagent, who is helping a visitor, to both show information to the visitorusing a screen sharing session and then to view the visitor's screenusing a co-browsing session, all without requiring the visitor todownload and install software, plugin, etc.

As noted above, in one embodiment an agent may show information to thevisitor by opening a screen sharing viewer in a div layered on top ofthe visitor's browser page, and cause another portion of the website oranything else on the agent's screen to be visible within the screensharing viewer.

FIG. 22 shows an example screen shot of a co-browse agent view 2200according to an embodiment. As shown in FIG. 22, the agent view includesan icon 2210 in the co-browse agent view header 2220. The co-browseagent view 2200 also includes a field 2230 showing the content of thevisitor's browser.

FIG. 23 shows a popup 2310 that is generated when the agent clicks onthe icon 2210. As shown in FIG. 23, the popup 2310 includes a menu ofoptions, including an option to allow the agent to show the agent'sscreen 2320 and an option to allow the agent to view the visitor'sscreen 2330. FIG. 24 shows the visitor's view if the agent selects theoption to show the agent's screen 2320, and FIGS. 25-27 show a sequenceof views if the if the agent selects the option to view the visitor'sscreen 2330.

FIG. 24 shows the visitor's view if the agent selects the option to showthe agent's screen 2320. As shown in FIG. 24, when the agent initiates ascreen sharing session and opts to show the contents of the agent'sscreen, a view of the agent's screen 2400 will be overlayed on top ofthe visitor's browsing session 2410. The view of the agent's screen maycontain information from any source desired by the agent. Accordingly,the agent may opt to include a view of the agent's screen, a videosource such as a quick time video source of an attached device, imagesfrom an attached camera, or any other available video stream. Althoughnot shown in FIG. 24, upon selecting the option to show the agent'sscreen the agent may be provided with a dialog through which the agentis allowed to select the source of video to be provided to the visitorover the screen sharing session. Although the view of the agent's screenis rendered in a div layered on top of the visitor's page, thescreenshare view is not included in the agent's co-browse viewer, toavoid confusion where there agent would be seeing an image of his ownscreen.

FIGS. 25-27 show a sequence of views which occur at the visitor side ifthe agent elects to initiate a screen sharing session in which thevisitor's screen will be shown to the agent (button 2330 in FIG. 23).Starting a screen sharing session, in one embodiment, requires thevisitor's screen to be captured using screen sharing software. Anexample screen sharing software that may be used for this purpose iscommercially available from Glance Networks, although the invention isnot limited to an implementation that uses Glance Networks screensharing software.

Screen sharing software may need to be installed. Accordingly, as shownin FIG. 25, if the visitor does not have screen sharing softwareinstalled, the visitor will be provided with a dialog 2500 prompting thevisitor to download the screen sharing software client. If the visitorclicks on link 2510 the screen sharing software client download willinitiate. If the visitor already has the screen sharing software clientdownloaded, the step shown in FIG. 25 is not necessary and is skipped.

FIG. 26 shows a view of the visitor's screen once the screen sharingsoftware client has been downloaded to the visitor, and when a screensharing session is to be started on which the visitor's screen will beshown to the agent. As shown in FIG. 26, the visitor is prompted with anotification 2600 showing that the agent has initiated a screen sharingsession and enabling the visitor to either allow the screen sharingsession 2610 or decline the screen sharing session 2620. If the visitorclicks “decline” 2620 the screen sharing session is terminated. If thevisitor clicks “accept” 2610, the screen sharing session is initiatedand the visitor's screen will be captured and transmitted to the agentto be viewed by the agent.

FIG. 27 shows the visitor's view of their screen if the visitor clicks“accept 2610” once a screen sharing session has been initiated in whichthe agent is viewing the visitor's screen. As shown in FIG. 27, acolored border 2700 is placed around the visitor's screen to indicate tothe visitor that the visitor is sharing the content of what is shown onthe display with the agent. A dialog 2710 is also provided through whichthe visitor is able to control the screen sharing session. For example,in box 2720 of dialog 2710 the visitor is provided with options such asto allow the agent to have remote control over the visitor's computer,stop remote control by the agent, end the screen sharing session, etc.

FIG. 28 shows the agent's view of the visitor's screen when a screensharing session showing the visitor's screen has been initiated. Asshown in FIG. 28, the agent is able to see the same content of thevisitor's screen as the visitor is able to see. Rather than simplyseeing the content of the visitor's browser, as was available during theco-browsing session, use of a screen sharing session enables the visitorto see more of the visitor's computer display by including aspects ofthe display in addition to the visitor's browser. Further, since thescreen sharing software enables remote control in which the personviewing the screen sharing software is able direct operation of thehost's computer, the agent is able to use the screen sharing session tonot only see the visitor's screen but also control operation of thevisitor's computer. As noted above, remote control by the agent is aprivilege granted by the visitor, so the visitor has the option toresume control of their computer at any point during the screen sharingsession.

Secret Co-Browse

It is possible to have the JavaScript launch a co-browse sessionautomatically without the knowledge of the visitor. This enables peoplemonitoring the site to watch how visitors use the site to determinewhich links may be difficult to find and which aspects of the site mightcause confusion. Likewise, website monitors or agents who are monitoringa website may view co-browsing sessions of visitors to determine what aparticular visitor is looking at on the web site and choose toproactively initiate contact with the visitor if desired.

In one embodiment a service such as a chat service tracks visitors asthey visit a web site. In this embodiment, the chat service (e.g. LivePerson) tags the visitor with a unique identifying number (key). TheAgent is provided with a list of visitors as well as the keys that tagthose visitors. JavaScript (cobrowse.js) downloaded from the web sitecauses the visitor's browser to poll a server for any session startedthat is associated with the key.

If the agent decides to participate in a co-browsing session with avisitor, the agent selects the visitor from a list or otherwiseinstructs software on the agent system to initiate a co-browsing sessionassociated with the key. The key is then passed to a co-browse serverwhich notifies the polling server that a co-browse session associatedwith the key has been initiated.

When the visitor polls the polling server, the polling server willreturn a positive value to the visitor which will cause the cobrowse.jsto provide browser state on the co-browsing session to the co-browseserver. The agent waits till the visitor connects to the co-browsingsession and then the agent connects.

In this embodiment, the visitor does not to take any action to passinformation to the agent. Rather, the tracking server, in this instanceLivePerson, assigns a tracking value (key) to the visitor and providesthis key to the agent. JavaScript provided by the web site is used tocause the visitor browser to poll for sessions associated with the key,and similarly co-browse JavaScript is used to cause the visitor's webbrowser to participate in the co-browsing session. The visitor does notneed to do anything to participate in the co-browsing session andlikewise likely will not know that what is they see in their browserduring their browsing session is also being seen by an agent associatedwith the web site. This has the further advantage of enabling the agentto start the session without requiring the user to do anything—i.e.without requiring the user to click on a particular tab or other featureof the website. Likewise, the service could also record the sessions formarketing intelligence, training, law enforcement, and other purposes.

The methods described herein may be implemented as software configuredto be executed in control logic such as contained in a CPU of anelectronic device such as a computer. The functions described herein maybe implemented as sets of program instructions that are stored in anon-transitory tangible computer readable medium. When implemented inthis manner, the computer program includes a set of instructions which,when executed by a computer, cause the computer to perform a method toimplement the functions described above. Programmable logic can be fixedtemporarily or permanently in a non-transitory tangible computerreadable medium such as a read-only memory chip, a computer memory, adisk, or other storage medium. In addition to being implemented insoftware, the logic described herein can be embodied using discretecomponents, integrated circuitry, programmable logic used in conjunctionwith a programmable logic device such as a Field Programmable Gate Array(FPGA) or microprocessor, or any other device including any combinationthereof. All such embodiments are intended to fall within the scope ofthe present invention.

Various changes and modifications of the embodiments shown in thedrawings and described in the specification may be made within thespirit and scope of the present invention. Accordingly, it is intendedthat all matter contained in the above description and shown in theaccompanying drawings be interpreted in an illustrative and not in alimiting sense.

What is claimed is: 1-68. (canceled)
 69. A method of selecting aco-browse session associated with a visitor to a web site from aplurality of co-browse sessions maintained by a co-browse connectionserver, the method comprising: identifying a browser associated with thevisitor; and correctly identifying which of the plurality of co-browsesessions is associated with the identified browser.
 70. The method ofclaim 69, wherein identifying the browser comprises instructing thevisitor to send a signal to the co-browse connection server by clickinga button on a web page displayed in the browser.
 71. The method of claim70, wherein clicking the button supplies a random session ID to thevisitor, the method further comprising the step of receiving the randomsession ID from the visitor and using the random session ID in the stepof correctly identifying the co-browse session associated with theidentified browser.
 72. The method of claim 71, wherein the step ofcorrectly identifying the co-browse session associated with theidentified visitor comprises providing, by the agent, the random sessionID to the co-browse connection server.
 73. The method of claim 71,wherein the step of correctly identifying the co-browse sessionassociated with the identified browser comprises selecting, by theagent, a co-browse session from a list of co-browse sessions using thereceived random session ID to select from the list.
 74. The method ofclaim 69, wherein identifying the browser associated with the visitorcomprises enabling the visitor to enter identifying information inconnection with initiating a co-browse session, and using theidentifying information to select the co-browse session from theplurality of co-browse sessions.
 75. The method of claim 74, wherein theidentifying information comprises personal information about thevisitor, including the visitor's name, account ID, or email address. 76.The method of claim 69, further comprising running Customer RelationshipManagement (CRM) software, and wherein the step of correctly identifyingwhich of the co-browse sessions is associated with the identifiedbrowser comprises selecting a record for the visitor by the CRMsoftware.
 77. The method of claim 76, further comprising enabling thevisitor to enter identifying information in connection with initiatingthe co-browse session, and wherein the step of selecting the record inthe CRM software is implemented in response to detecting that thevisitor has entered identifying information.
 78. The method of claim 77,wherein the step of selecting the record causes a notification to theagent via the CRM software.
 79. The method of claim 77, furthercomprising creating a new CRM software record for a visitor if the CRMsoftware does not already have a record for the visitor when the visitorenters identifying information in connection with initiating theco-browse session.
 80. The method of claim 76, wherein the CRM softwarehas a list showing current visitors to the website.
 81. The method ofclaim 76, wherein the CRM software record for the visitor contains abutton to enable the agent to join a co-browse session with the visitorassociated with the CRM software record.
 82. The method of claim 69,further comprising a chat service providing the ability of the visitorand the agent to engage in a text or voice based chat conversation. 83.The method of claim 82, further comprising the step of causing thevisitor to complete a pre-chat survey before the step of initiating achat session between the agent and the visitor.
 84. The method of claim83, further comprising the step of using information from the pre-chatsurvey to identify a co-browse session associated with the visitor. 85.The method of claim 84, wherein the step of using information compriseslooking up the visitor information in the CRM software system
 86. Themethod of claim 69, wherein the website contains chat service scriptenabling a chat service to initiate chat sessions between agents andvisitors.
 87. The method of claim 86, wherein the co-browsing script isincluded with the chat service script.
 88. The method of claim 87,wherein both the co-browsing scrip and chat service script aredownloaded to the browser from the chat service.
 89. The method of claim86, wherein the step of identifying the browser associated with thevisitor is implemented via the chat script, and
 90. The method of claim86, wherein the step of correctly identifying which of the plurality ofco-browse sessions is associated with the identified browser isimplemented via information obtained from the visitor via the chatscript.
 91. The method of claim 90, further comprising the step ofinitiating a co-browsing session by the agent with the correctlyidentified co-browsing session after assuming responsibility for thechat session with the visitor.
 92. The method of claim 91, wherein thestep of initiating the co-browsing session by the agent does not requirethe visitor to take action or provide permission prior in connectionwith establishing the co-browsing session.
 93. The method of claim 92,wherein the visitor does not know the agent has initiated theco-browsing session.
 94. The method of claim 86, wherein the co-browsescript includes a co-browse session identifying value uniquelyassociated with a chat session between the agent and visitor.
 95. Themethod of claim 94, wherein the identifying value is used in the step ofcorrectly identifying which of the plurality of co-browse sessions isassociated with the identified browser to join a co-browsing sessionwith the visitor. 96-124. (canceled)